Adaptive Context Encoding Module for Semantic Segmentation

The object sizes in images are diverse, therefore, capturing multiple scale context information is essential for semantic segmentation. Existing context aggregation methods such as pyramid pooling module (PPM) and atrous spatial pyramid pooling (ASPP) design different pooling size or atrous rate, such that multiple scale information is captured. However, the pooling sizes and atrous rates are chosen manually and empirically. In order to capture object context information adaptively, in this paper, we propose an adaptive context encoding (ACE) module based on deformable convolution operation to argument multiple scale information. Our ACE module can be embedded into other Convolutional Neural Networks (CNN) easily for context aggregation. The effectiveness of the proposed module is demonstrated on Pascal-Context and ADE20K datasets. Although our proposed ACE only consists of three deformable convolution blocks, it outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets. All the experiment study confirms that our proposed module is effective as compared to the state-of-the-art methods

Residual Networks based Distortion Classification and Ranking for Laparoscopic Image Quality Assessment

Laparoscopic images and videos are often affected by different types of distortion like noise, smoke, blur and nonuniform illumination. Automatic detection of these distortions, followed generally by application of appropriate image quality enhancement methods, is critical to avoid errors during surgery. In this context, a crucial step involves an objective assessment of the image quality, which is a two-fold problem requiring both the classification of the distortion type affecting the image and the estimation of the severity level of that distortion. Unlike existing image quality measures which focus mainly on estimating a quality score, we propose in this paper to formulate the image quality assessment task as a multi-label classification problem taking into account both the type as well as the severity level (or rank) of distortions. Here, this problem is then solved by resorting to a deep neural networks based approach. The obtained results on a laparoscopic image dataset show the efficiency of the proposed approach.Comment: 5 Pages, ICIP 202

CD-COCO: A Versatile Complex Distorted COCO Database for Scene-Context-Aware Computer Vision

The recent development of deep learning methods applied to vision has enabled their increasing integration into real-world applications to perform complex Computer Vision (CV) tasks. However, image acquisition conditions have a major impact on the performance of high-level image processing. A possible solution to overcome these limitations is to artificially augment the training databases or to design deep learning models that are robust to signal distortions. We opt here for the first solution by enriching the database with complex and realistic distortions which were ignored until now in the existing databases. To this end, we built a new versatile database derived from the well-known MS-COCO database to which we applied local and global photo-realistic distortions. These new local distortions are generated by considering the scene context of the images that guarantees a high level of photo-realism. Distortions are generated by exploiting the depth information of the objects in the scene as well as their semantics. This guarantees a high level of photo-realism and allows to explore real scenarios ignored in conventional databases dedicated to various CV applications. Our versatile database offers an efficient solution to improve the robustness of various CV tasks such as Object Detection (OD), scene segmentation, and distortion-type classification methods. The image database, scene classification index, and distortion generation codes are publicly available \footnote{\url{https://github.com/Aymanbegh/CD-COCO}

A Novel Architectural Framework on IoT Ecosystem, Security Aspects and Mechanisms: A Comprehensive Survey

For the past few years, the Internet of Things (IoT) technology continues to not only gain popularity and importance, but also witnesses the true realization of everything being smart. With the advent of the concept of smart everything, IoT has emerged as an area of great potential and incredible growth. An IoT ecosystem centers around innovation perspective which is considered as its fundamental core. Accordingly, IoT enabling technologies such as hardware and software platforms as well as standards become the core of the IoT ecosystem. However, any large-scale technological integration such as the IoT development poses the challenge to ensure secure data transmission. Perhaps, the ubiquitous and the resource-constrained nature of IoT devices and the sensitive and private data being generated by IoT systems make them highly vulnerable to physical and cyber threats. In this paper, we re-define an IoT ecosystem from the core technologies view point. We propose a modified three layer IoT architecture by dividing the perception layer into elementary blocks based on their attributed functions. Enabling technologies, attacks and security countermeasures are classified under each layer of the proposed architecture. Additionally, to give the readers a broader perspective of the research area, we discuss the role of various state-of-the-art emerging technologies in the IoT security. We present the security aspects of the most prominent standards and other recently developed technologies for IoT which might have the potential to form the yet undefined IoT architecture. Among the technologies presented in this article, we give a special interest to one recent technology in IoT domain. This technology is named IQRF that stands for Intelligent Connectivity using Radio Frequency. It is an emerging technology for wireless packet-oriented communication that operates in sub-GHz ISM band (868 MHz) and which is intended for general use where wireless connectivity is needed, either in a mesh network or point-to-point (P2P) configuration. We also highlighted the security aspects implemented in this technology and we compare it with the other already known technologies. Moreover, a detailed discussion on the possible attacks is presented. These attacks are projected on the IoT technologies presented in this article including IQRF. In addition, lightweight security solutions, implemented in these technologies, to counter these threats in the proposed IoT ecosystem architecture are also presented. Lastly, we summarize the survey by listing out some common challenges and the future research directions in this field.publishedVersio

A Neural Network based Framework for Effective Laparoscopic Video Quality Assessment

Video quality assessment is a challenging problem having a critical significance in the context of medical imaging. For instance, in laparoscopic surgery, the acquired video data suffers from different kinds of distortion that not only hinder surgery performance but also affect the execution of subsequent tasks in surgical navigation and robotic surgeries. For this reason, we propose in this paper neural network-based approaches for distortion classification as well as quality prediction. More precisely, a Residual Network (ResNet) based approach is firstly developed for simultaneous ranking and classification task. Then, this architecture is extended to make it appropriate for the quality prediction task by using an additional Fully Connected Neural Network (FCNN). To train the overall architecture (ResNet and FCNN models), transfer learning and end-to-end learning approaches are investigated. Experimental results, carried out on a new laparoscopic video quality database, have shown the efficiency of the proposed methods compared to recent conventional and deep learning based approaches

Can Image Enhancement be Beneficial to Find Smoke Images in Laparoscopic Surgery?

Laparoscopic surgery has a limited field of view. Laser ablation in a laproscopic surgery causes smoke, which inevitably influences the surgeon's visibility. Therefore, it is of vital importance to remove the smoke, such that a clear visualization is possible. In order to employ a desmoking technique, one needs to know beforehand if the image contains smoke or not, to this date, there exists no accurate method that could classify the smoke/non-smoke images completely. In this work, we propose a new enhancement method which enhances the informative details in the RGB images for discrimination of smoke/non-smoke images. Our proposed method utilizes weighted least squares optimization framework~(WLS). For feature extraction, we use statistical features based on bivariate histogram distribution of gradient magnitude~(GM) and Laplacian of Gaussian~(LoG). We then train a SVM classifier with binary smoke/non-smoke classification task. We demonstrate the effectiveness of our method on Cholec80 dataset. Experiments using our proposed enhancement method show promising results with improvements of 4\% in accuracy and 4\% in F1-Score over the baseline performance of RGB images. In addition, our approach improves over the saturation histogram based classification methodologies Saturation Analysis~(SAN) and Saturation Peak Analysis~(SPA) by 1/5\% and 1/6\% in accuracy/F1-Score metrics.Comment: In proceedings of IST, Color and Imaging Conference (CIC 26). Congcong Wang and Vivek Sharma contributed equally to this work and listed in alphabetical orde

Towards a Video Quality Assessment based Framework for Enhancement of Laparoscopic Videos

Laparoscopic videos can be affected by different distortions which may impact the performance of surgery and introduce surgical errors. In this work, we propose a framework for automatically detecting and identifying such distortions and their severity using video quality assessment. There are three major contributions presented in this work (i) a proposal for a novel video enhancement framework for laparoscopic surgery; (ii) a publicly available database for quality assessment of laparoscopic videos evaluated by expert as well as non-expert observers and (iii) objective video quality assessment of laparoscopic videos including their correlations with expert and non-expert scores.Comment: SPIE Medical Imaging 2020 (Draft version